Lessons Learned and Future Directions of MetaTutor: Leveraging Multichannel Data to Scaffold Self-Regulated Learning With an Intelligent Tutoring System.

Self-regulated learning (SRL) is critical for learning across tasks, domains, and contexts. Despite its importance, research shows that not all learners are equally skilled at accurately and dynamically monitoring and regulating their self-regulatory processes. Therefore, learning technologies, such as intelligent tutoring systems (ITSs), have been designed to measure and foster SRL. This paper presents an overview of over 10 years of research on SRL with MetaTutor, a hypermedia-based ITS designed to scaffold college students' SRL while they learn about the human circulatory system. MetaTutor's architecture and instructional features are designed based on models of SRL, empirical evidence on human and computerized tutoring principles of multimedia learning, Artificial Intelligence (AI) in educational systems for metacognition and SRL, and research on SRL from our team and that of other researchers. We present MetaTutor followed by a synthesis of key research findings on the effectiveness of various versions of the system (e.g., adaptive scaffolding vs. no scaffolding of self-regulatory behavior) on learning outcomes. First, we focus on findings from self-reports, learning outcomes, and multimodal data (e.g., log files, eye tracking, facial expressions of emotion, screen recordings) and their contributions to our understanding of SRL with an ITS. Second, we elaborate on the role of embedded pedagogical agents (PAs) as external regulators designed to scaffold learners' cognitive and metacognitive SRL strategy use. Third, we highlight and elaborate on the contributions of multimodal data in measuring and understanding the role of cognitive, affective, metacognitive, and motivational (CAMM) processes. Additionally, we unpack some of the challenges these data pose for designing real-time instructional interventions that scaffold SRL. Fourth, we present existing theoretical, methodological, and analytical challenges and briefly discuss lessons learned and open challenges.

Tracking Changes in Students' Online Self-Regulated Learning Behaviors and Achievement Goals Using Trace Clustering and Process Mining.

Success in online and blended courses requires engaging in self-regulated learning (SRL), especially for challenging STEM disciplines, such as physics. This involves students planning how they will navigate course assignments and activities, setting goals for completion, monitoring their progress and content understanding, and reflecting on how they completed each assignment. Based on Winne & Hadwin's COPES model, SRL is a series of events that temporally unfold during learning, impacted by changing internal and external factors, such as goal orientation and content difficulty. Thus, as goal orientation and content difficulty change throughout a course, so might students' use of SRL processes. This paper studies how students' SRL behavior and achievement goal orientation change over time in a large (N = 250) college introductory level physics course taught online. Students' achievement goal orientation was measured by repeated administration of the achievement goals questionnaire-revised (AGQ-R). Students' SRL behavior was measured by analyzing their clickstream event traces interacting with online learning modules via a combination of trace clustering and process mining. Event traces were first divided into groups similar in nature using agglomerative clustering, with similarity between traces determined based on a set of derived characteristics most reflective of students' SRL processes. We then generated causal nets for each cluster of traces via process mining and interpreted the underlying behavior and strategy of each causal net according to the COPES SRL framework. We then measured the frequency at which students adopted each causal net and assessed whether the adoption of different causal nets was associated with responses to the AGQ-R. By repeating the analysis for three sets of online learning modules assigned at the beginning, middle, and end of the semester, we examined how the frequency of each causal net changed over time, and how the change correlated with changes to the AGQ-R responses. Results have implications for measuring the temporal nature of SRL during online learning, as well as the factors impacting the use of SRL processes in an online physics course. Results also provide guidance for developing online instructional materials that foster effective SRL for students with different motivational profiles.

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties.

Learning disabilities (LDs) encompass disorders of those who have difficulty learning and using academic skills, exhibiting performance below expectations for their chronological age in the areas of reading, writing, and/or mathematics. Each of the disorders making up the LDs involve different deficits; however, some commonalities can be found within that heterogeneity, such in terms of learning self-regulation and metacognition. Unlike in early ages and later educational levels, there are hardly any evidence-based evaluation protocols for adults with LDs. LDs influence academic performance but also have serious consequences in professional, social, and family contexts. In response to this, the current work proposes a multimodal evaluation protocol focused on metacognitive, self-regulation of learning, and emotional processes, which make up the basis of the difficulties in adults with LDs. The assessment is carried out through analysis of the on-line learning process using a variety methods, techniques, and sensors (e.g., eye tracking, facial expressions of emotion, physiological responses, concurrent verbalizations, log files, screen recordings of human-machine interactions) and off-line methods (e.g., questionnaires, interviews, and self-report measures). This theoretically-driven and empirically-based guideline aims to provide an accurate assessment of LDs in adulthood in order to design effective prevention and intervention proposals.

Multiple Negative Emotions During Learning With Digital Learning Environments - Evidence on Their Detrimental Effect on Learning From Two Methodological Approaches.

Emotions are a core factor of learning. Studies have shown that multiple emotions are co-experienced during learning and have a significant impact on learning outcomes. The present study investigated the importance of multiple, co-occurring emotions during learning about human biology with MetaTutor, a hypermedia-based tutoring system. Person-centered as well as variable-centered approaches of cluster analyses were used to identify emotion clusters. The person-centered clustering analyses indicated three emotion profiles: a positive, negative and neutral profile. Students with a negative profile learned less than those with other profiles and also reported less usage of emotion regulation strategies. Emotion patterns identified through spectral co-clustering confirmed these results. Throughout the learning activity, emotions built a stable correlational structure of a positive, a negative, a neutral and a boredom emotion pattern. Positive emotion pattern scores before the learning activity and negative emotion pattern scores during the learning activity predicted learning, but not consistently. These results reveal the importance of negative emotions during learning with MetaTutor. Potential moderating factors and implications for the design and development of educational interventions that target emotions and emotion regulation with digital learning environments are discussed.